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Program Description 1 

Developmental summer bridge programs are designed to reduce the 
need for developmental education in college by providing students 
with accelerated instruction in areas where additional knowledge and 
skills are needed to help them succeed in higher education. These 
programs occur in the summer “bridge” period between high school 
and college. 2 Summer bridge programs are based on theories and 
empirical evidence suggesting that the transition to college can be 
difficult for academically underprepared students. Thus, providing 
accelerated academic and college knowledge skills during this 
bridge period can presumably improve students’ college readiness. 3 
Developmental bridge programs most often incorporate two compo- 
nents: accelerated developmental instruction addressing identified 
academic deficiencies, typically in reading, writing, and/or math; and 
college preparation training designed to ease students’ emotional 
and psychological adjustment to the college environment. The college 
preparation training component of summer bridge programs often 
includes information on financial aid, study skills training, time 
management, computer literacy, and college culture. 

Research 4 

The What Works Clearinghouse (WWC) identified one study of developmental summer bridge programs that both 
falls within the scope of the Interventions for Developmental Students in Postsecondary Education topic area and 
meets WWC group design standards. This study met WWC group design standards without reservations. The study 
included 1 ,318 undergraduate students across eight colleges in Texas. 

The WWC considers the extent of evidence for developmental summer bridge programs to be small for three 
outcome domains— academic achievement, postsecondary enrollment, and credit accumulation. (See the 
Effectiveness Summary on p. 5 for more details of effectiveness by domain.) 
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Effectiveness 

Developmental summer bridge programs were found to have no discernible effects on academic achievement, 
postsecondary enrollment, and credit accumulation for postsecondary students. 


Table 1. Summary of findings 5 




Improvement index (percentile points) 




Outcome domain 

Rating of effectiveness 

Average 

Range 

Number of 
studies 

Number of 
students 

Extent of 
evidence 

Academic 

achievement 

No discernible effects 

+2 

+1 to +3 

1 

1,318 

Small 

Postsecondary 

enrollment 

No discernible effects 

+1 

na 

1 

1,318 

Small 

Credit accumulation 

No discernible effects 

-1 

-2 toO 

1 

1,318 

Small 


na = not applicable 
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Program Information 

Background 

There are many models of summer bridge programs, with content varying widely across the institutions that imple- 
ment them and the student populations being served. Historically, summer bridge programs have targeted ethnic/ 
racial minority, low-income, first-generation, or other student populations deemed at risk of dropping out of college. 
This intervention report, however, concentrates solely on developmental summer bridge programs— that is, programs 
with a core emphasis on providing developmental education for students who are identified as unprepared for 
college-level work in one or more classes. These developmental summer bridge programs explicitly target students 
whose scores on college placement tests indicate the need for remediation. 

Developmental summer bridge programs offer the opportunity for students to complete required developmental 
coursework during the summer before the first year of college, so that they may enroll in college-level coursework in 
the fall of their first year of college. This is important given that student retention is negatively related to the amount 
of time students are required to spend in developmental education courses, for which they are typically required 
to pay tuition but do not earn credits toward their degree. 6 Developmental summer bridge programs also aim to 
ease the transition to college by socializing students to institutional expectations of which they may be unaware or 
unequipped to master (e.g., study skills, time management skills, how to interact with faculty). These nonacademic 
college readiness components are designed to provide cultural and social capital to students and promote adjust- 
ment to college culture. 

Program details 

The developmental summer bridge programs that were implemented across the eight community colleges included 
in the one study reviewed in this intervention report were highly similar. They were all offered during summer 2009 
and took place at two open-admissions 4-year institutions and six community colleges in Texas. The programs 
were delivered between 3-6 hours a day across the span of 4-5 weeks. All students received accelerated academic 
instruction designed to address deficiencies in math, reading, and/or writing. Students also received academic 
support services that typically involved tutoring, mentoring, and/or computer-based learning labs. Students also 
received training related to college preparation skills, which involved both academic (study and test-taking skills, 
time management, career assessment, financial aid) and social components (financial responsibility, motivation, 
behavioral expectations, stress management). Finally, students were given the opportunity to earn a stipend of 
up to $400; typically, students received $150 at the beginning of the program and $250 after successful completion 
of the program. 


Cost 

Barnett, Bork, Mayer, Pretlow, Wathington, and Weiss (2012) report on implementation costs across the eight com- 
munity colleges included in the one study reviewed in this intervention report. Implementation costs varied across 
the eight community colleges, ranging from a low of $835 per student to a high of $2,349 per student (expressed 
in 2009 dollars). The average cost across the eight colleges was $1 ,319 per student (with a standard deviation of 
$502). Approximately 32% of the implementation costs were attributed to staffing, and 27% were associated with 
stipend incentives offered to students (up to $400 per student, with an average of $354 per student). All of these 
costs were due solely to the supplemental nature of the program beyond practice as usual at the colleges. Barnett 
et al. (2012) also reported that there was no strong evidence of economies of scale related to the number of students 
enrolled, given that the student stipends were constant for each additional student served. 

Barnett et al. (2012) also explored the cost effectiveness of the developmental bridge programs implemented in the 
study included in this intervention report, a discussion of which is beyond the scope of a WWC review. 7 
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Research Summary 

The WWC identified 22 studies that investigated the effects of develop- Table 2. Scope of reviewed research 
mental summer bridge programs on the enrollment and achievement of 
postsecondary students. 

The WWC reviewed all 22 of those studies against group design stan- 
dards. One study, reported in four different manuscripts (Barnett, Bork, 

Mayer, Pretlow, Wathington, & Weiss, 2012; Pretlow, 201 1 ; Wathington, 

Barnette, Weissman, Teres, Pretlow, & Nakanishi, 201 1 ; Wathington, Pretlow, & Mitchell, 2011) is a randomized controlled 
trial that meets WWC group design standards without reservations. That study is summarized in this report. Nine studies 
do not meet WWC group design standards. The remaining 12 studies do not meet WWC eligibility screens for review in 
this topic area. Citations for all 22 studies are included in the References section, which begins on p. 8. 

Summary of study meeting WWC group design standards without reservations 

Barnett et al. (2012) conducted a randomized controlled trial with 1 ,322 students at eight colleges in Texas: El Paso 
Community College, Lone Star City College-CyFair, Lone Star College-Kingwood, Palo Alto College, San Antonio 
College, St. Philip’s College, South Texas College, and Texas A&M International University. The program group 
included 799 eligible students who were randomly assigned to participate in a developmental summer bridge 
program in the summer before their first semester of enrollment. Although the program content varied across 
the eight sites, all programs included accelerated instruction in math, reading, and/or writing; academic support 
(e.g., through tutoring, mentoring, and/or access to learning labs); college knowledge support and training; and 
the opportunity to earn a $400 stipend. The comparison group included 533 students who were not invited to 
participate in the developmental summer bridge programs but who were free to pursue any other summer activities 
that the colleges provided. Follow-up data were collected on 1 ,318 students up to 2 years after randomization. 

Summary of studies meeting WWC group design standards with reservations 

No studies of developmental summer bridge programs met WWC group design standards with reservations. 


Grade 

Postsecondary 

Delivery method 

Whole class 

Program type 

Practice 
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Effectiveness Summary 

The WWC review of developmental summer bridge programs for the Interventions for Developmental Students in 
Postsecondary Education topic area includes student outcomes in five domains: academic achievement, post- 
secondary enrollment, credit accumulation, degree attainment, and progress in developmental education. The one 
study of developmental summer bridge programs that meets WWC group design standards reported findings in 
three of the five domains: (a) academic achievement (b) postsecondary enrollment, and (c) credit accumulation. The 
findings below present the authors’ estimates and WWC-calculated estimates of the size and statistical significance 
of the effects of developmental summer bridge programs on developmental students’ outcomes in postsecondary 
education. For a more detailed description of the rating of effectiveness and extent of evidence criteria, see the 
WWC Rating Criteria on p. 22. 

Summary of effectiveness for the academic achievement domain 

One study that met WWC group design standards without reservations reported findings in the academic 
achievement domain. 

Barnett et al. (2012) reported on whether students passed their first college-level math course, first college-level 
reading course, and first college-level writing course at the 2-year follow-up. The authors reported no statistically 
significant differences between the intervention and comparison groups in terms of the percentage of students 
who passed their first college-level math course (47% vs. 43%), first college-level reading course (73% vs. 72%), 
and first college-level writing course (72% vs. 68%). None of these differences were large enough to be considered 
substantively important. The WWC characterizes the mean effect across the three measures in this domain as an 
indeterminate effect. 

The study authors also reported effects separately by gender and students’ maternal educational attainment. The 
authors reported no statistically significant gender differences in program effects on the percentage of students 
who passed their first college-level math course (47% vs. 43% for women in the program and comparison groups; 
46% vs. 43% for men in the program and comparison groups), first college-level reading course (73% for women 
in both groups; 73% vs. 70% for men in the program and comparison groups), and first college-level writing course 
(73% vs. 69% for women in the program and comparison groups; 71 % vs. 68% for men in the program and 
comparison groups). The authors reported no statistically significant differences in program effects by maternal 
education in terms of the percentage of students who passed their first college-level math course (53% vs. 51 % 
with maternal college attendance in the program and comparison groups; 43% vs. 37% for those with no maternal 
college attendance in the program and comparison groups), first college-level reading course (82% vs. 80% with 
maternal college attendance in the program and comparison groups; 68% for those with no maternal college atten- 
dance in both groups), and first college-level writing course (76% vs. 74% with maternal college attendance 
in the program and comparison groups; 69% vs. 65% for those with no maternal college attendance in the program 
and comparison groups). None of these differences were large enough to be considered substantively important. 

Thus, for the academic achievement domain, the study that met WWC group design standards without reservations 
did not show either a statistically significant effect or an effect large enough to be considered substantively important. 
This results in a rating of no discernible effects, with a small extent of evidence. 

Supplementary findings in the academic achievement domain, which were measured at interim follow-up periods, 
are reported in Appendix D. 
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Table 3. Rating of effectiveness and extent of evidence for the academic achievement domain 


Rating of effectiveness 

Criteria met 

No discernible effects 

None of the studies show 
statistically significant or 
substantively important effects, 
either positive or negative. 

In the one study that reported findings, the estimated impact of the intervention on outcomes in the academic 
achievement domain was neither statistically significant nor large enough to be substantively important. 

Extent of evidence 

Criteria met 

Small 

One study that included 1 ,318 students in eight colleges reported evidence of effectiveness in the academic 
achievement domain. 


Summary of effectiveness for the postsecondary enrollment domain 

One study that met WWC group design standards without reservations reported findings in the postsecondary 
enrollment domain. 

Barnett et al. (2012) found no statistically significant differences in the percentage of intervention and comparison group 
students who were registered at any college at the first-semester follow-up (82% vs. 81 %), and this effect was not large 
enough to be considered substantively important. The WWC characterizes this finding as an indeterminate effect. 

Thus, for the postsecondary enrollment domain, the study that met WWC group design standards without reservations 
did not show either a statistically significant effect or an effect large enough to be considered substantively important. 
This results in a rating of no discernible effects, with a small extent of evidence. 


Table 4. Rating of effectiveness and extent of evidence for the postsecondary enrollment domain 


Rating of effectiveness 

Criteria met 

No discernible effects 

None of the studies show 
statistically significant or 
substantively important effects, 
either positive or negative. 

In the one study that reported findings, the estimated impact of the intervention on outcomes in the postsecondary 
enrollment domain was neither statistically significant nor large enough to be substantively important. 

Extent of evidence 

Criteria met 

Small 

One study that included 1 ,318 students in eight colleges reported evidence of effectiveness in the postsecondary 
enrollment domain. 


Summary of effectiveness for the credit accumulation domain 

One study that met WWC group design standards without reservations reported findings in the credit accumula- 
tion domain. 

Barnett et al. (2012) reported on the percentage of students registered at any college, the number of semesters 
registered at any college, and the number of college-level credits earned at the 2-year follow-up. The authors 
reported no statistically significant differences between the intervention and comparison groups on the percentage 
of students registered at any college (64% vs. 66%), the number of semesters registered (3.3 vs. 3.4), or the number 
of college-level credits earned (15.9 in both groups). None of these differences were large enough to be considered 
substantively important. The WWC characterizes the mean effect across the three measures in this domain as an 
indeterminate effect. 
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The study authors also reported effects separately by gender and students’ maternal educational attainment. 

The authors reported no statistically significant gender differences in program effects on number of semesters 
registered (3.3 for women in both groups; 3.3 vs. 3.5 for men in the program and comparison groups). The authors 
reported no statistically significant gender differences in program effects on number of college-level credits earned 
(15.7 vs. 15.2 for women in the program and comparison groups; 16.6 vs. 16.7 for men in the program and comparison 
groups). The authors reported no statistically significant differences in program effects by maternal education in terms 
of number of semesters registered (3.6 for those with maternal college attendance in both groups; 3.2 vs. 3.4 for those 
with no maternal college attendance in the program and comparison groups), and number of college-level credits 
earned (18.3 vs. 19.2 with maternal college attendance in the program and comparison groups; 14.4 vs. 14.5 for 
those with no maternal college attendance in the program and comparison groups). None of these differences were 
large enough to be considered substantively important. 

Thus, for the credit accumulation domain, the study that met WWC group design standards without reservations 
did not show either a statistically significant effect or an effect large enough to be considered substantively important. 
This results in a rating of no discernible effects, with a small extent of evidence. 

Supplemental findings in the credit accumulation domain, which were measured at interim follow-up periods, 
are reported in Appendix D. 


Table 5. Rating of effectiveness and extent of evidence for the credit accumulation domain 


Rating of effectiveness 

Criteria met 

No discernible effects 

None of the studies show 
statistically significant or 
substantively important effects, 
either positive or negative. 

In the one study that reported findings, the estimated impact of the intervention on outcomes in the credit 
accumulation domain was neither statistically significant nor large enough to be substantively important. 

Extent of evidence 

Criteria met 

Small 

One study that included 1 ,318 students in eight colleges reported evidence of effectiveness in the credit 
accumulation domain. 
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Appendix A: Research details for Barnett et al. (2012) 

Barnett, E. A., Bork, R. H., Mayer, A. K., Pretlow, J., Wathington, H. D., & Weiss, M. J. (2012). Bridging 
the gap: An impact study of eight developmental summer bridge programs in Texas. New York: 
National Center for Postsecondary Research. http://files.eric.ed.gov/fulltext/ED539188.pdf. 

Table A. Summary of findings Meets WWC group design standards without reservations 


Study findings 
Average improvement index 


Outcome domain 

Sample size 

(percentile points) 

Statistically significant 

Academic achievement 

1,318 students 

+2 

No 

Postsecondary enrollment 

1,318 students 

+1 

No 

Credit accumulation 

1,318 students 

-1 

No 


Setting The study took place at eight colleges in Texas: El Paso Community College, Lone Star City 
College-CyFair, Lone Star College-Kingwood, Palo Alto College, San Antonio College, St. 
Philip’s College, South Texas College, and Texas A&M International University. 

Study sample The developmental summer bridge programs targeted students whose scores on a college 

placement test indicated the need for remediation in at least one academic subject addressed 
in the college’s summer bridge program (math, reading, and/or writing). The study enrolled 
students who applied for admission into a developmental summer bridge program and 
consented to participate in the study. Randomization was conducted at the student level at 
each of the eight colleges. Across all sites, a total of 1 ,332 students were recruited, and 1 ,31 8 
students were included in the final analytic sample. More than half of the students (n=799) 
were randomly assigned to the intervention group, and the rest (n=533) were randomly assigned 
to the comparison group. Demographically, more than half (62%) of the entire sample of 
study participants were female. The majority of participants (84%) were Hispanic, 9% were 
non-Hispanic White, and 7% were non-Hispanic African American. Most participants (85%) 
were 18 or younger at the start of the study. Further, 29% indicated receiving some form of 
public assistance, and 41 % reported being the first in their family to attend college. 8 


Intervention The developmental summer bridge programs were implemented in the summer before postsec- 
group ondary enrollment and delivered daily over the course of 4-5 weeks. The daily sessions lasted 
3-6 hours. Across all sites, the hours of instruction varied from 35 to 100 hours, with most 
institutions providing more than 52 hours of total instruction. Although the specific content of 
the developmental summer bridge programs varied across sites, all of them involved accelerated 
academic instruction in at least one area of academic need (math, reading, writing), academic 
support services, guidance about academic expectations and knowledge needed to navigate 
college, and the opportunity to receive a stipend of $400 (with $150 delivered at the beginning 
of the program and $250 delivered on successful completion of the program). Four of the colleges 
implemented a course-based program that involved standard developmental college courses 
condensed to fit within the summer time frame, where students received developmental education 
credits upon successful completion of the program. The other four colleges used a freestanding 
program, which did not follow an established developmental college course structure, such that 
students were not required to enroll in developmental courses during the summer and therefore 
did not receive developmental education credits for participation in the summer bridge program. 
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Comparison 

group 


Outcomes and 
measurement 


Support for 
implementation 


Students assigned to the comparison group were allowed to pursue any other summer activities 
that the college provided, including enrollment in developmental or college-level summer 
courses. Of the students assigned to the comparison group, 32% reported that they expected 
to take summer classes (at any institution), and 16% did in fact enroll in summer courses at 
the institution to which they applied for admission to the summer bridge program. Therefore, 
the comparison group received standard services in the summer before enrollment. 

Researchers reported outcomes at five points in time: the fall semester after the program 
(fall 2009), the spring semester after the program (spring 201 0), the summer semester after 
the program (summer 201 0), the second fall semester after the program (fall 201 0), and the 
second spring semester after the program (spring 201 1 , 2 years after random assignment). 

As specified in the Interventions for Developmental Students in Postsecondary Education 
review protocol, for outcomes measured at multiple follow-up periods, the longest follow-up 
period was selected as the primary outcome of interest, with findings from earlier time points 
included in supplemental tables (Appendix D). However, as specified in the protocol, the first 
measure of enrollment was selected as the primary outcome of interest in the postsecondary 
enrollment outcome domain. Measures of enrollment occurring after the first semester of 
college were categorized under the credit accumulation domain, and the longest follow-up 
period was selected as the primary outcome within that outcome domain. For a more detailed 
description of the outcome measures, see Appendix B. 

The Texas Higher Education Coordinating Board provided funding to the eight colleges to 
establish and implement the developmental summer bridge programs. The program evaluator, 
the National Center for Postsecondary Research, provided the funds to cover the cost of students’ 
stipends. All eight programs were reported to be implemented with reasonable fidelity to the 
program model. Programs ranged in size from 52 to 165 students, who attended the program 
3-6 hours a day, 4-5 days per week, for 4-5 weeks. 
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Appendix B: Outcome measures for each domain 


Academic achievement 

Passed college-level math 

Taken from Texas Higher Education Coordinating Board records, this outcome assesses whether students 
passed the first college-level (i.e., non-developmental) math course taken. 

Passed college-level reading 

Taken from Texas Higher Education Coordinating Board records, this outcome assesses whether students 
passed the first college-level (i.e., non-developmental) reading course taken. 

Passed college-level writing 

Taken from Texas Higher Education Coordinating Board records, this outcome assesses whether students 
passed the first college-level (i.e., non-developmental) writing/English course taken. 

Postsecondary enrollment 

First-semester registration 
at any college 

Taken from school administrative and Texas Higher Education Coordinating Board records, this outcome 
assesses whether students were registered at any college in Texas the first semester after the bridge 
program was offered. 

Credit accumulation 

Registered at any college 

Taken from school administrative and Texas Higher Education Coordinating Board records, this outcome 
assesses whether students were registered at any college in Texas subsequent to their first semester after 
the bridge program was offered. 

Semesters registered at any college 

Taken from school administrative and Texas Higher Education Coordinating Board records, this outcome 
assesses the number of semesters students were registered at any college in Texas. 

College-level credits earned 

Taken from school administrative records, this outcome assesses the total number of college-level 
(i.e., non-developmental) credits students earned. 
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Appendix C.1: Findings included in the rating for the academic achievement domain 


Mean 

(standard deviation) WWC calculations 

Study Sample Intervention Comparison Mean Effect Improvement 


Outcome measure 

sample 

size 

group 

group 

difference 

size 

index 

p-value 

Barnett et al., 2012 a 

Passed college-level math 

College 

students 

1,318 

47% 

43% 

4% 

0.07 

3 

.19 

Passed college-level reading 

College 

students 

1,318 

73% 

72% 

1% 

0.02 

1 

.68 

Passed college-level writing 

College 

students 

1,318 

72% 

68% 

4% 

0.07 

3 

.18 

Domain average for academic achievement (Barnett et al., 2012) 



0.06 

2 

Not 

statistically 

significant 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on individual outcomes, representing the average change expected for all individuals 
who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the 
change in an average individual’s percentile rank that can be expected if the individual is given the intervention. The WWC-computed average effect size is a simple average 
rounded to two decimal places; the average improvement index is calculated from the average effect size. The statistical significance of the study's domain average was determined 
by the WWC. 

a For Barnett et al. (2012), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-values presented here were 
reported in the original study. Proportions presented for the outcomes are the posttest percentages for each group passing college-level courses (provided by the study authors). 
Effect sizes are computed as a Cox’s index: logged-odds ratio transformation divided by 1 .65. Please see the WWC Standards and Procedures Handbook (version 3.0) for the 
computation of effect sizes for binary outcomes. This study is characterized as having indeterminate effects because the reported effect size was neither statistically significant 
nor large enough to be substantively important. For more information, please refer to the WWC Standards and Procedures Handbook (version 3.0), p. 26. 


Appendix C.2: Findings included in the rating for the postsecondary enrollment domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Barnett et al., 2012 a 

First-semester registration 
at any college 

College 

students 

1,318 

82% 

81% 

1% 

0.02 

1 

.67 


Domain average for postsecondary enrollment (Barnett etal., 2012) 0.02 1 na 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on individual outcomes, representing the average change expected for all individuals 
who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the 
change in an average individual’s percentile rank that can be expected if the individual is given the intervention, na = not applicable. 

a For Barnett et al. (2012), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-value presented here was reported 
in the original study. Proportions presented for first-semester registration are the posttest percentages for each group (provided by the study authors). Effect size is computed as a 
Cox's index: logged-odds ratio transformation divided by 1 .65. Please see the WWC Standards and Procedures Handbook (version 3.0) for the computation of effect sizes for binary 
outcomes. This study is characterized as having indeterminate effects because the reported effect size was neither statistically significant nor large enough to be substantively 
important. For more information, please refer to the WWC Standards and Procedures Handbook (version 3.0), p. 26. 
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Appendix C.3: Findings included in the rating for the credit accumulation domain 


Mean 

(standard deviation) WWC calculations 



Study 

Sample 

Intervention 

Comparison 

Mean 

Effect 

Improvement 


Outcome measure 

sample 

size 

group 

group 

difference 

size 

index 

p-value 

Barnett et al., 2012 a 

Registered at any college 

College 

students 

1,318 

64% 

66% 

-2% 

-0.04 

-2 

.49 

Semesters registered at 

College 

1,318 

3.3 

3.4 

-0.10 

-0.06 

-2 

.37 

any college 

students 


(nr) 

(nr) 





College-level credits earned 

College 

1,318 

15.9 

15.9 

0 

0 

0 

.97 


students 


(nr) 

(nr) 






Domain average for credit accumulation (Barnett et al., 2012) -0.03 -1 Not 

statistically 

significant 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on individual outcomes, representing the average change expected for all individu- 
als who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting 
the change in an average individual's percentile rank that can be expected if the individual is given the intervention. The WWC-computed average effect size is a simple average 
rounded to two decimal places; the average improvement index is calculated from the average effect size. The statistical significance of the study's domain average was deter- 
mined by the WWC. nr = not reported. 

a For Barnett et al. (2012), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-values presented here were 
reported in the original study. Proportions presented for registered at any college are the posttest percentages for each group (provided by the study authors). Effect size for registered 
at any college is computed as a Cox's index; logged-odds ratio transformation divided by 1 .65. Please see the WWC Standards and Procedures Handbook (version 3.0) for the com- 
putation of effect sizes for binary outcomes. This study is characterized as having indeterminate effects because the reported effect size was neither statistically significant nor large 
enough to be substantively important. For more information, please refer to the WWC Standards and Procedures Handbook (version 3.0), p. 26. 
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Appendix D.1: Description of supplemental findings for the academic achievement domain 


Mean 

(standard deviation) WWC calculations 


Outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Barnett et al., 2012 a 

Passed college-level math 

Women 

810 

47% 

43% 

4% 

0.08 

3 

.28 

Passed college-level math 

Men 

489 

46% 

43% 

3% 

0.07 

3 

.46 

Passed college-level reading 

Women 

810 

73% 

73% 

0% 

-0.01 

0 

.86 

Passed college-level reading 

Men 

489 

73% 

70% 

3% 

0.07 

3 

.43 

Passed college-level writing 

Women 

810 

73% 

69% 

3% 

0.07 

3 

.32 

Passed college-level writing 

Men 

489 

71% 

68% 

3% 

0.08 

3 

.41 

Passed college-level math 

Mother with 
some college 

426 

53% 

51% 

2% 

0.04 

2 

.67 

Passed college-level math 

Mother with 
no college 

755 

43% 

37% 

6% 

0.13 

5 

.09 

Passed college-level reading 

Mother with 
some college 

426 

82% 

80% 

2% 

0.06 

2 

.57 

Passed college-level reading 

Mother with 
no college 

755 

68% 

68% 

0% 

-0.02 

-1 

.83 

Passed college-level writing 

Mother with 
some college 

426 

76% 

74% 

3% 

0.06 

2 

.52 

Passed college-level writing 

Mother with 
no college 

755 

69% 

65% 

4% 

0.08 

3 

.28 

Passed college-level math 

First semester 
follow-up 

1,318 

11% 

5% 

6% 

0.22 

9 

.00 

Passed college-level reading 

First semester 
follow-up 

1,318 

32% 

29% 

3% 

0.08 

3 

.17 

Passed college-level writing 

First semester 
follow-up 

1,318 

33% 

29% 

4% 

0.09 

4 

.10 

Passed college-level math 

Second 

semester 

follow-up 

1,318 

32% 

23% 

9% 

0.22 

9 

.00 

Passed college-level reading 

Second 

semester 

follow-up 

1,318 

63% 

58% 

5% 

0.11 

4 

.06 

Passed college-level writing 

Second 

semester 

follow-up 

1,318 

63% 

58% 

5% 

0.11 

4 

.06 

Passed college-level math 

1 -year 
follow-up 

1,318 

36% 

28% 

8% 

0.17 

7 

.00 

Passed college-level reading 

1 -year 
follow-up 

1,318 

65% 

61% 

4% 

0.08 

3 

.14 

Passed college-level writing 

1-year 

follow-up 

1,318 

65% 

60% 

5% 

0.11 

5 

.04 
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Passed college-level math 

Third 

semester 

follow-up 

1,318 

43% 

37% 

6% 

0.12 

5 

.03 

Passed college-level reading 

Third 

semester 

follow-up 

1,318 

70% 

67% 

3% 

0.08 

3 

.14 

Passed college-level writing 

Third 

semester 

follow-up 

1,318 

69% 

65% 

6% 

0.10 

4 

.08 


Table Notes: The supplemental findings presented in this table are additional findings from studies in this report that do not factor into the determination of the intervention rating. 
For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors the comparison 
group. The effect size is a standardized measure of the effect of an intervention on individual outcomes, representing the average change expected for all individuals who are given 
the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change in an aver- 
age individual's percentile rank that can be expected if the individual is given the intervention. 

a For Barnett et al. (2012), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-values presented here were 
reported in the original study. Proportions presented for the outcomes are the posttest percentages for each group passing college-level courses (provided by the study authors). 
Effect sizes are computed as a Cox’s index: logged-odds ratio transformation divided by 1 .65. Please see the WWC Standards and Procedures Handbook (version 3.0) for the 
computation of effect sizes for binary outcomes. This study is characterized as having indeterminate effects because the reported effect size was neither statistically significant 
nor large enough to be substantively important. For more information, please refer to the WWC Standards and Procedures Handbook (version 3.0), p. 26. 
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Appendix D.2: Description of supplemental findings for the credit accumulation domain 


Mean 

(standard deviation) WWC calculations 



Study 

Sample 

Intervention 

Comparison 

Mean 

Effect 

Improvement 


Outcome measure 

sample 

size 

group 

group 

difference 

size 

index 

p-value 

Barnett et al., 2012 a 

Semesters registered at 

Women 

810 

3.3 

3.3 

0 

0 

0 

.96 

any college 



(nr) 

(nr) 





Semesters registered at 

Men 

489 

3.3 

3.5 

-0.20 

-0.18 

-7 

.18 

any college 



(nr) 

(nr) 





College-level credits earned 

Women 

810 

15.7 

15.2 

0.50 

0.03 

1 

.67 




(nr) 

(nr) 





College-level credits earned 

Men 

489 

16.6 

16.7 

-0.10 

-0.01 

0 

.92 




(nr) 

(nr) 





Semesters registered at 

Mother with 

426 

3.6 

3.6 

0 

0 

0 

.93 

any college 

some college 


(nr) 

(nr) 





Semesters registered at 

Mother with 

755 

3.2 

3.4 

-0.20 

-0.15 

-6 

.11 

any college 

no college 


(nr) 

(nr) 





College-level credits earned 

Mother with 

426 

18.3 

19.2 

-0.90 

-0.06 

-2 

.55 


some college 


(nr) 

(nr) 





College-level credits earned 

Mother with 

755 

14.4 

14.5 

-0.10 

-0.01 

0 

.98 


no college 


(nr) 

(nr) 





Semesters registered at 

First semester 

1,318 

0.8 

0.8 

0 

0 

0 

.67 

any college 

follow-up 


(nr) 

(nr) 





College-level credits earned 

First semester 

1,318 

4.0 

4.0 

0 

0 

0 

.74 


follow-up 


(nr) 

(nr) 





Registered at any college 

Second 

semester 

follow-up 

1,318 

79% 

80% 

-1% 

-0.01 

0 

.86 

Semesters registered at 

Second 

1,318 

1.6 

1.6 

0 

0 

0 

.89 

any college 

semester 

follow-up 


(nr) 

(nr) 





College-level credits earned 

Second 

1,318 

8.0 

7.8 

0.20 

0.02 

1 

.66 


semester 

follow-up 


(nr) 

(nr) 





Registered at any college 

1 year 
follow-up 

1,318 

40% 

46% 

-6% 

-0.12 

-5 

.03 

Semesters registered at 

1 year 

1,318 

2.0 

2.1 

-0.10 

-0.06 

-2 

.33 

any college 

follow-up 


(nr) 

(nr) 





College-level credits earned 

1 year 

1,318 

9.0 

9.2 

-0.20 

-0.02 

-1 

.81 


follow-up 


(nr) 

(nr) 
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Registered at any college 

Third 

semester 

follow-up 

1,318 

67% 

68% 

-1% 

-0.02 

-1 

.74 

Semesters registered at 

Third 

1,318 

2.7 

2.7 

0 

0 

0 

.39 

any college 

semester 

follow-up 


(nr) 

(nr) 





College-level credits earned 

Third 

1,318 

12.7 

12.5 

0.20 

0.02 

1 

.81 


semester 

follow-up 


(nr) 

(nr) 






Table Notes: The supplemental findings presented in this table are additional findings from studies in this report that do not factor into the determination of the intervention rating. 
For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors the comparison 
group. The effect size is a standardized measure of the effect of an intervention on individual outcomes, representing the average change expected for all individuals who are given 
the intervention (measured in standard deviations of the outcome measure). The improvement index is an alternate presentation of the effect size, reflecting the change in an aver- 
age individual's percentile rank that can be expected if the individual is given the intervention, nr = not reported. 

a For Barnett et al. (2012), no corrections for clustering or multiple comparisons and no difference-in-differences adjustments were needed. The p-values presented here were 
reported in the original study. Proportions presented for registered at any college are the posttest percentages for each group (provided by the study authors). Effect sizes for 
registered at any college are computed as a Cox's index: logged-odds ratio transformation divided by 1 .65. Please see the WWC Standards and Procedures Handbook (version 
3.0) for the computation of effect sizes for binary outcomes. This study is characterized as having indeterminate effects because the reported effect size was neither statistically 
significant nor large enough to be substantively important. For more information, please refer to the WWC Standards and Procedures Handbook (version 3.0), p. 26. 
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Endnotes 

1 The descriptive information for this program was obtained from Barnett et al. (2012), Kezar (2000), Sablan (2014), and Wathington et al. 
(201 1). The WWC requests that distributors review the program description sections for accuracy from their perspective. Further verifi- 
cation of the accuracy of the descriptive information for this program is beyond the scope of this review. 

2 Kezar, A. (2000). Summer bridge programs: Supporting all students. ERIC Digest. Washington, DC: ERIC Clearinghouse on Higher 
Education. http://files.eric.ed.gov/fulltext/ED442421 .pdf; Sablan, J. R. (2014). The challenge of summer bridge programs. American 
Behavioral Scientist, 58, 1035-1050. 

3 Conley, D. T. (2008). Rethinking college readiness. New Directions for Higher Education, 144, 3-13; Deil-Amen, R,, & Rosenbaum, J. E. 
(2003). The social prerequisites of success: Can college structure reduce the need for social know-how? The Annals of the American 
Academy of Political and Social Science, 586, 1 20-1 43; Venezia, A., & Hughes, K. L. (201 3). Acceleration Strategies in the new develop- 
mental education landscape. New Directions for Community Colleges, 1 64, 37-45. 

4 The literature search reflects documents publicly available by May 2014. The studies in this report were reviewed using the Standards 
from the WWC Procedures and Standards Handbook (version 3.0), along with those described in the Interventions for Developmental 
Students in Postsecondary Education review protocol (version 3.0). The evidence presented in this report is based on available 
research. Findings and conclusions may change as new research becomes available. 

5 The one study that met group design standards and is summarized in this intervention report did not report outcomes in the degree 
attainment domain, which is covered in the Interventions for Developmental Students in Postsecondary Education topic area protocol. 
The study did report results in the progress in developmental education domain (measured as number of developmental credits 
earned); however, outcomes in this domain were not included in the intervention report because the direction of expected effects 
would vary across the eight intervention sites depending on whether students earned developmental credits during the intervention 
period. 

6 Attewell, R, Lavin, D., Domina, T., & Levey, T. (2006). New evidence on college remediation. Journal of Higher Education, 77, 886-924. 

7 Interested readers are referred to Barnett et al. (2012), pp. 40-42. 

8 Pretlow (2011). 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2015, March). 
Developmental Students in Postsecondary Education intervention report: Developmental summer bridge 
programs. Retrieved from http://whatworks.ed.gov 
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WWC Rating Criteria 

Criteria used to determine the rating of a study 

Study rating 

Criteria 

Meets WWC group design 
standards without reservations 

A study that provides strong evidence for an intervention’s effectiveness, such as a well-implemented RCT. 

Meets WWC group design 

A study that provides weaker evidence for an intervention's effectiveness, such as a QED or an RCT with high 

standards with reservations 

attrition that has established equivalence of the analytic samples. 

Criteria used to determine the rating of effectiveness for an intervention 

Rating of effectiveness 

Criteria 

Positive effects 

Two or more studies show statistically significant positive effects, at least one of which met WWC evidence 
standards for a strong design, AND 

No studies show statistically significant or substantively important negative effects. 

Potentially positive effects 

At least one study shows a statistically significant or substantively important positive effect, AND 

No studies show a statistically significant or substantively important negative effect AND fewer or the same number 

of studies show indeterminate effects than show statistically significant or substantively important positive effects. 

Mixed effects 

At least one study shows a statistically significant or substantively important positive effect AND at least one study 
shows a statistically significant or substantively important negative effect, but no more such studies than the number 
showing a statistically significant or substantively important positive effect, OR 

At least one study shows a statistically significant or substantively important effect AND more studies show an 
indeterminate effect than show a statistically significant or substantively important effect. 

Potentially negative effects 

One study shows a statistically significant or substantively important negative effect and no studies show 
a statistically significant or substantively important positive effect, OR 

Two or more studies show statistically significant or substantively important negative effects, at least one study 
shows a statistically significant or substantively important positive effect, and more studies show statistically 
significant or substantively important negative effects than show statistically significant or substantively important 
positive effects. 

Negative effects 

Two or more studies show statistically significant negative effects, at least one of which met WWC evidence 
standards for a strong design, AND 

No studies show statistically significant or substantively important positive effects. 

No discernible effects 

None of the studies shows a statistically significant or substantively important effect, either positive or negative. 

Criteria used to determine the extent of evidence for an intervention 

Extent of evidence 

Criteria 

Medium to large 

The domain includes more than one study, AND 
The domain includes more than one school, AND 

The domain findings are based on a total sample size of at least 350 students, OR, assuming 25 students in a class, 
a total of at least 14 classrooms across studies. 

Small 

The domain includes only one study, OR 
The domain includes only one school, OR 

The domain findings are based on a total sample size of fewer than 350 students, AND, assuming 25 students 
in a class, a total of fewer than 14 classrooms across studies. 
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Glossary of Terms 

Attrition 

Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Extent of evidence 

Improvement index 

Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Rating of effectiveness 

Single-case design 
Standard deviation 


Statistical significance 


Substantively important 


Attrition occurs when an outcome variable is not available for all participants initially assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review and inclusion in this report if it falls within the scope of the 
review protocol and uses either an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

An indication of how much evidence supports the findings. The criteria for the extent 
of evidence levels are given in the WWC Rating Criteria on p. 21 

Along a percentile distribution of individuals, the improvement index represents the gain 
or loss of the average individual due to the intervention. As the average individual starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which study participants are 
assigned to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which eligible study participants are 
randomly assigned to intervention and comparison groups. 

The WWC rates the effects of an intervention in each domain based on the quality of the 
research design and the magnitude, statistical significance, and consistency in findings. The 
criteria for the ratings of effectiveness are given in the WWC Rating Criteria on p. 21 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample tend to be spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < .05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 3.0) for additional details. 
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